ebab7d0da53c51438720d60c6e2e467c - /page/wikibook/entry.mu

Micron Document

<!DOCTYPE html>

<title>Entropy coding</title>

</head>

<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Entropy coding</span></span>

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style>

<p>In <a href="Information_theory" title="Information theory">information theory</a>, an <b>entropy coding</b> (or <b>entropy encoding</b>) is any lossless data compression method that attempts to approach the lower bound declared by <a href="Claude_Shannon" title="Claude Shannon">Shannon's</a> <a href="Source_coding_theorem" class="mw-redirect" title="Source coding theorem">source coding theorem</a>, which states that any lossless data compression method must have an expected code length greater than or equal to the entropy of the source.<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup>

</p><p>More precisely, the source coding theorem states that for any source distribution, the expected code length satisfies <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \operatorname {E} _{x\sim P}[\ell (d(x))]\geq \operatorname {E} _{x\sim P}[-\log _{b}(P(x))]}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \operatorname {E} _{x\sim P}[\ell (d(x))]\geq \operatorname {E} _{x\sim P}[-\log _{b}(P(x))]}</annotation>

</semantics>

</math></span><img src="./0f94727ba75e7f47c300c86f901598e22dfedd60.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:36.154ex; height:2.843ex;" alt="{\displaystyle \operatorname {E} _{x\sim P}[\ell (d(x))]\geq \operatorname {E} _{x\sim P}[-\log _{b}(P(x))]}" loading="lazy"></span>, where <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ell }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ell }</annotation>

</semantics>

</math></span><img src="./f066e981e530bacc07efc6a10fa82deee985929e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.97ex; height:2.176ex;" alt="{\displaystyle \ell }" loading="lazy"></span> is the function specifying the number of symbols in a code word, <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle d}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle d}</annotation>

</semantics>

</math></span><img src="./e85ff03cbe0c7341af6b982e47e9f90d235c66ab.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.216ex; height:2.176ex;" alt="{\displaystyle d}" loading="lazy"></span> is the coding function, <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle b}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle b}</annotation>

</semantics>

</math></span><img src="./f11423fbb2e967f986e36804a8ae4271734917c3.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:0.998ex; height:2.176ex;" alt="{\displaystyle b}" loading="lazy"></span> is the number of symbols used to make output codes and <span class="mwe-math-element mwe-math-element-inline"><span class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"><math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle P}">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle P}</annotation>

</semantics>

</math></span><img src="./b4dc73bf40314945ff376bd363916a738548d40a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.745ex; height:2.176ex;" alt="{\displaystyle P}" loading="lazy"></span> is the probability of the source symbol. An entropy coding attempts to approach this lower bound.

</p><p>Two of the most common entropy coding techniques are <a href="Huffman_coding" title="Huffman coding">Huffman coding</a> and <a href="Arithmetic_coding" title="Arithmetic coding">arithmetic coding</a>.<sup id="cite_ref-Huffman_1952_pp._1098–1101_2-0" class="reference"><a href="#cite_note-Huffman_1952_pp._1098–1101-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup>

If the approximate entropy characteristics of a data stream are known in advance (especially for <a href="Signal_compression" title="Signal compression">signal compression</a>), a simpler static code may be useful.

These static codes include <a href="Universal_code_(data_compression)" title="Universal code (data compression)">universal codes</a> (such as <a href="Elias_gamma_coding" title="Elias gamma coding">Elias gamma coding</a> or <a href="Fibonacci_coding" title="Fibonacci coding">Fibonacci coding</a>) and <a href="Golomb_coding" title="Golomb coding">Golomb codes</a> (such as <a href="Unary_coding" title="Unary coding">unary coding</a> or <a href="Rice_coding" class="mw-redirect" title="Rice coding">Rice coding</a>).

</p><p>Since 2014, data compressors have started using the <a href="Asymmetric_numeral_systems" title="Asymmetric numeral systems">asymmetric numeral systems</a> family of entropy coding techniques, which allows combination of the compression ratio of <a href="Arithmetic_coding" title="Arithmetic coding">arithmetic coding</a> with a processing cost similar to <a href="Huffman_coding" title="Huffman coding">Huffman coding</a>.

</p>

<div class="mw-heading mw-heading2"><h2 id="Entropy_as_a_measure_of_similarity">Entropy as a measure of similarity</h2></div>

<p>Besides using entropy coding as a way to compress digital data, an entropy encoder can also be used to measure the amount of <a href="Similarity_measure" title="Similarity measure">similarity</a> between <a href="Data_stream" title="Data stream">streams of data</a> and already existing classes of data. This is done by generating an entropy coder/compressor for each class of data; unknown data is then <a href="Statistical_classification" title="Statistical classification">classified</a> by feeding the uncompressed data to each compressor and seeing which compressor yields the highest compression. The coder with the best compression is probably the coder trained on the data that was most similar to the unknown data.

</p>

<ul><li><a href="Arithmetic_coding" title="Arithmetic coding">Arithmetic coding</a></li>

<li><a href="Asymmetric_numeral_systems" title="Asymmetric numeral systems">Asymmetric numeral systems</a> (ANS)</li>

<li><a href="Context-adaptive_binary_arithmetic_coding" title="Context-adaptive binary arithmetic coding">Context-adaptive binary arithmetic coding</a> (CABAC)</li>

<li><a href="Huffman_coding" title="Huffman coding">Huffman coding</a></li>

<li><a href="Range_coding" title="Range coding">Range coding</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFDudaTahboubGadgilDelp2015" class="citation book cs1">Duda, Jarek; Tahboub, Khalid; Gadgil, Neeraj J.; Delp, Edward J. (May 2015). <a rel="nofollow" class="external text" href="https://ieeexplore.ieee.org/document/7170048/;jsessionid=wI8UmMMsBsrAYcNN1Ux2gHW5ReVUf0HhrXFHlaZZVSmlJdKWDIv8!-2026188604">"The use of asymmetric numeral systems as an accurate replacement for Huffman coding"</a>. <i>2015 Picture Coding Symposium (PCS)</i>. pp. <span class="nowrap">65–</span>69. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FPCS.2015.7170048">10.1109/PCS.2015.7170048</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4799-7783-3</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:20260346">20260346</a>.</cite></span>

</li>

<li id="cite_note-Huffman_1952_pp._1098–1101-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-Huffman_1952_pp._1098–1101_2-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFHuffman1952" class="citation journal cs1">Huffman, David (1952). "A Method for the Construction of Minimum-Redundancy Codes". <i>Proceedings of the IRE</i>. <b>40</b> (9). Institute of Electrical and Electronics Engineers (IEEE): <span class="nowrap">1098–</span>1101. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Fjrproc.1952.273898">10.1109/jrproc.1952.273898</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0096-8390">0096-8390</a>.</cite></span>

</li>

</ol></div></div>

<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>

<ul><li><i><a rel="nofollow" class="external text" href="http://www.inference.phy.cam.ac.uk/mackay/itila/book.html">Information Theory, Inference, and Learning Algorithms</a></i>, by <a href="David_MacKay_(scientist)" class="mw-redirect" title="David MacKay (scientist)">David MacKay</a> (2003), gives an introduction to Shannon theory and data compression, including the <a href="Huffman_coding" title="Huffman coding">Huffman coding</a> and <a href="Arithmetic_coding" title="Arithmetic coding">arithmetic coding</a>.</li>

<li><i><a rel="nofollow" class="external text" href="http://iphome.hhi.de/wiegand/assets/pdfs/VBpart1.pdf">Source Coding</a>,</i> by <a href="Thomas_Wiegand" title="Thomas Wiegand">T. Wiegand</a> and H. Schwarz (2011).</li></ul>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}

/* end https://en.wikipedia.org/ */

</style><style data-mw-deduplicate="TemplateStyles:r1236075235">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style></div><div role="navigation" class="navbox" aria-labelledby="Data_compression_methods241" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><style data-mw-deduplicate="TemplateStyles:r1239400231">

/* start https://en.wikipedia.org/ */

.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div id="Data_compression_methods241" style="font-size:114%;margin:0 4em"><a href="Data_compression" title="Data compression">Data compression</a> methods</div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Lossless_compression" title="Lossless compression">Lossless</a><br>type</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%"></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Adaptive_coding" title="Adaptive coding">Adaptive coding</a></li>

<li><a href="Arithmetic_coding" title="Arithmetic coding">Arithmetic</a></li>

<li><a href="Asymmetric_numeral_systems" title="Asymmetric numeral systems">Asymmetric numeral systems</a></li>

<li><a href="Golomb_coding" title="Golomb coding">Golomb</a></li>

<li><a href="Huffman_coding" title="Huffman coding">Huffman</a>

<ul><li><a href="Adaptive_Huffman_coding" title="Adaptive Huffman coding">Adaptive</a></li>

<li><a href="Canonical_Huffman_code" title="Canonical Huffman code">Canonical</a></li>

<li><a href="Modified_Huffman_coding" title="Modified Huffman coding">Modified</a></li></ul></li>

<li><a href="Range_coding" title="Range coding">Range</a></li>

<li><a href="Shannon_coding" title="Shannon coding">Shannon</a></li>

<li><a href="Shannon%E2%80%93Fano_coding" title="Shannon–Fano coding">Shannon–Fano</a></li>

<li><a href="Shannon%E2%80%93Fano%E2%80%93Elias_coding" title="Shannon–Fano–Elias coding">Shannon–Fano–Elias</a></li>

<li><a href="Tunstall_coding" title="Tunstall coding">Tunstall</a></li>

<li><a href="Unary_coding" title="Unary coding">Unary</a></li>

<li><a href="Universal_code_(data_compression)" title="Universal code (data compression)">Universal</a>

<ul><li><a href="Exponential-Golomb_coding" title="Exponential-Golomb coding">Exp-Golomb</a></li>

<li><a href="Fibonacci_coding" title="Fibonacci coding">Fibonacci</a></li>

<li><a href="Elias_gamma_coding" title="Elias gamma coding">Gamma</a></li>

<li><a href="Levenshtein_coding" title="Levenshtein coding">Levenshtein</a></li></ul></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Dictionary_coder" title="Dictionary coder">Dictionary</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Byte-pair_encoding" title="Byte-pair encoding">Byte-pair encoding</a></li>

<li><a href="LZ77_and_LZ78" title="LZ77 and LZ78">Lempel–Ziv</a>

<li><a href="Snappy_(compression)" title="Snappy (compression)">Snappy</a></li></ul></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Other</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<li><a href="Delta_encoding" title="Delta encoding">Delta</a>

<ul><li><a href="Incremental_encoding" title="Incremental encoding">Incremental</a></li></ul></li>

<li><a href="Grammar-based_code" title="Grammar-based code">Grammar</a>

<li><a href="Sequitur_algorithm" title="Sequitur algorithm">Sequitur</a></li></ul></li>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Hybrid</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li>LZ77 + Huffman

<ul><li><a href="Deflate" title="Deflate">Deflate</a></li>

<li>LZ77 + ANS

<ul><li><a href="LZFSE" title="LZFSE">LZFSE</a></li></ul></li>

<li>LZ77 + Huffman + ANS

<ul><li><a href="Zstd" title="Zstd">Zstandard</a></li></ul></li>

<li>LZ77 + Huffman + context

<ul><li><a href="Brotli" title="Brotli">Brotli</a></li></ul></li>

<li>LZSS + Huffman

<li>LZ77 + Range

<li>LZHAM</li></ul></li>

<li>RLE + BWT + MTF + Huffman

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Lossy_compression" title="Lossy compression">Lossy</a><br>type</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Transform_coding" title="Transform coding">Transform</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Discrete_cosine_transform" title="Discrete cosine transform">Discrete cosine transform</a>

<li><a href="Wavelet_transform" title="Wavelet transform">Wavelet</a>

<ul><li><a href="Daubechies_wavelet" title="Daubechies wavelet">Daubechies</a></li>

<li><a href="Set_partitioning_in_hierarchical_trees" title="Set partitioning in hierarchical trees">SPIHT</a></li></ul></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Predictive</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Adaptive_differential_pulse-code_modulation" title="Adaptive differential pulse-code modulation">ADPCM</a></li></ul></li>

<ul><li><a href="Algebraic_code-excited_linear_prediction" title="Algebraic code-excited linear prediction">ACELP</a></li>

<li>Motion

<ul><li><a href="Motion_compensation" title="Motion compensation">Compensation</a></li>

<li><a href="Motion_estimation" title="Motion estimation">Estimation</a></li>

<li><a href="Motion_vector" class="mw-redirect" title="Motion vector">Vector</a></li></ul></li>

<li><a href="Psychoacoustics" title="Psychoacoustics">Psychoacoustic</a></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Data_compression#Audio" title="Data compression">Audio</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Concepts</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<li><a href="Companding" title="Companding">Companding</a></li>

<li><a href="Convolution" title="Convolution">Convolution</a></li>

<li><a href="Dynamic_range" title="Dynamic range">Dynamic range</a></li>

<li><a href="Latency_(audio)" title="Latency (audio)">Latency</a></li>

<li><a href="Nyquist%E2%80%93Shannon_sampling_theorem" title="Nyquist–Shannon sampling theorem">Nyquist–Shannon theorem</a></li>

<li><a href="Sampling_(signal_processing)" title="Sampling (signal processing)">Sampling</a></li>

<li><a href="Silence_compression" title="Silence compression">Silence compression</a></li>

<li><a href="Sound_quality" title="Sound quality">Sound quality</a></li>

<li><a href="Speech_coding" title="Speech coding">Speech coding</a></li>

<li><a href="Sub-band_coding" title="Sub-band coding">Sub-band coding</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Audio_codec" title="Audio codec">Codec</a><br>parts</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Adaptive_differential_pulse-code_modulation" title="Adaptive differential pulse-code modulation">ADPCM</a></li>

<ul><li><a href="Algebraic_code-excited_linear_prediction" title="Algebraic code-excited linear prediction">ACELP</a></li>

<li><a href="Psychoacoustics" title="Psychoacoustics">Psychoacoustic model</a></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Image_compression" title="Image compression">Image</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Concepts</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Chroma_subsampling" title="Chroma subsampling">Chroma subsampling</a></li>

<li><a href="Coding_tree_unit" title="Coding tree unit">Coding tree unit</a></li>

<li><a href="Color_space" title="Color space">Color space</a></li>

<li><a href="Compression_artifact" title="Compression artifact">Compression artifact</a></li>

<li><a href="Image_resolution" title="Image resolution">Image resolution</a></li>

<li><a href="Macroblock" title="Macroblock">Macroblock</a></li>

<li><a href="Pixel" title="Pixel">Pixel</a></li>

<li><a href="Quantization_(image_processing)" title="Quantization (image processing)">Quantization</a></li>

<li><a href="Standard_test_image" title="Standard test image">Standard test image</a></li>

<li><a href="Texture_compression" title="Texture compression">Texture compression</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Methods</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Chain_code" title="Chain code">Chain code</a></li>

<li><a href="Deflate" title="Deflate">Deflate</a></li>

<li><a href="Fractal_compression" title="Fractal compression">Fractal</a></li>

<li><a href="Wavelet_transform" title="Wavelet transform">Wavelet</a>

<ul><li><a href="Daubechies_wavelet" title="Daubechies wavelet">Daubechies</a></li>

<li><a href="Set_partitioning_in_hierarchical_trees" title="Set partitioning in hierarchical trees">SPIHT</a></li></ul></li></ul>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Data_compression#Video" title="Data compression">Video</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Concepts</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<li><a href="Display_resolution" title="Display resolution">Display resolution</a></li>

<li><a href="Film_frame" title="Film frame">Frame</a></li>

<li><a href="Frame_rate" title="Frame rate">Frame rate</a></li>

<li><a href="Video_compression_picture_types" title="Video compression picture types">Frame types</a></li>

<li><a href="Interlaced_video" title="Interlaced video">Interlace</a></li>

<li><a href="Video#Characteristics_of_video_streams" title="Video">Video characteristics</a></li>

<li><a href="Video_quality" title="Video quality">Video quality</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Video_codec" title="Video codec">Codec</a><br>parts</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<li><a href="Deblocking_filter" title="Deblocking filter">Deblocking filter</a></li>

<li><a href="Lapped_transform" title="Lapped transform">Lapped transform</a></li>

<li>Motion

<ul><li><a href="Motion_compensation" title="Motion compensation">Compensation</a></li>

<li><a href="Motion_estimation" title="Motion estimation">Estimation</a></li>

<li><a href="Motion_vector" class="mw-redirect" title="Motion vector">Vector</a></li></ul></li>

<li><a href="Wavelet_transform" title="Wavelet transform">Wavelet</a>

<ul><li><a href="Daubechies_wavelet" title="Daubechies wavelet">Daubechies</a></li>

</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Information_theory" title="Information theory">Theory</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Compressed_data_structure" title="Compressed data structure">Compressed data structures</a>

<ul><li><a href="Compressed_suffix_array" title="Compressed suffix array">Compressed suffix array</a></li>

<li><a href="FM-index" title="FM-index">FM-index</a></li></ul></li>

<li><a href="Entropy_(information_theory)" title="Entropy (information theory)">Entropy</a></li>

<li><a href="Information_theory" title="Information theory">Information theory</a>

<ul><li><a href="Timeline_of_information_theory" title="Timeline of information theory">Timeline</a></li></ul></li>

<li><a href="Kolmogorov_complexity" title="Kolmogorov complexity">Kolmogorov complexity</a></li>

<li><a href="Prefix_code" title="Prefix code">Prefix code</a></li>

<li><a href="Quantization_(signal_processing)" title="Quantization (signal processing)">Quantization</a></li>

<li><a href="Rate%E2%80%93distortion_theory" title="Rate–distortion theory">Rate–distortion</a></li>

<li><a href="Redundancy_(information_theory)" title="Redundancy (information theory)">Redundancy</a></li>

<li><a href="Data_compression_symmetry" title="Data compression symmetry">Symmetry</a></li>

<li><a href="Smallest_grammar_problem" title="Smallest grammar problem">Smallest grammar problem</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Community</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Hutter_Prize" title="Hutter Prize">Hutter Prize</a></li></ul>

</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">People</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">

<ul><li><a href="Mark_Adler" title="Mark Adler">Mark Adler</a></li>

</div></td></tr></tbody></table></div>

<div class="navbox-styles"></div><div role="navigation" class="navbox authority-control" aria-label="Navbox391" style="padding:3px"><table class="nowraplinks hlist navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Authority control databases: National </th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"><ul><li><span class="uid"><a rel="nofollow" class="external text" href="https://d-nb.info/gnd/4743864-2">Germany</a></span></li></ul></div></td></tr></tbody></table></div></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-06-18" href="https://en.wikipedia.org/wiki/?title=Entropy_coding&oldid=1296201541">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>